Network-accessible control of one or more media devices

ABSTRACT

A content management (CM) system is provided to centrally control operation of one or more connected devices by issuing control requests and/or data requests. In some situations the connected device(s) include devices that control presentation of television programming-related content, such as digital video recorder (“DVR”) devices and media center devices, and in other situations may include other types of media devices and/or other electromechanical (“E/M”) devices that may be centrally controlled, such as E/M devices for home automation that may operate independently or in conjunction with the media devices. In some situations, the CM system communicates with the connected devices via a network and using appropriate protocol(s), and may be accessed remotely via a network so as to allow a user to remotely operate the CM system. The CM system may further control searching, identification, selection, and presentation of pieces of media content by the connected device(s).

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a divisional of U.S. patent application Ser.No. 11/120,656 filed May 2, 2005, now U.S. Pat. No. 7,792,920 entitled“Network-Accessible Control of One or More Media Devices,” which claimsthe benefit under 35 U.S.C. §119(e) of U.S. Provisional Patentapplication Ser. No. 60/567,177, entitled “Browser For MultimediaContent” and filed Apr. 30, 2004, and of U.S. Provisional PatentApplication Ser. No. 60/667,836, entitled “Interface For ManipulatingMultimedia Content” and filed Apr. 1, 2005, each of which isincorporated herein by reference in its entirety.

The present application is also related to each of U.S. patentapplication Ser. No. 11/120,635 filed May 2, 2005, entitled “ControllingOne Or More Media Devices;” U.S. patent application Ser. No. 11/120,654filed May 2, 2005, entitled “Controlling Content Presentation;” and U.S.patent application Ser. No. 11/120,653 filed May 2, 2005, entitled“Smart Home Control Of Electronic Devices,” each of which is filedconcurrently and is hereby incorporated by reference in its entirety.

TECHNICAL FIELD

The present disclosure generally relates to techniques for controllingmedia devices and/or other types of electronic devices.

BACKGROUND INFORMATION

Consumers in today's society are provided with many different types ofmultimedia content. Indeed, it is often possible for any particularpiece of information to be represented in the form of audio, television,text, video, still image, Internet Web page, or other multimedia format.Given the large amount of information available to consumers and giventhe many different multimedia formats available, it is beneficial toassist consumers in identifying, obtaining, and presenting pieces ofmultimedia content that may be of interest.

In the current world of television, movies, and related media systems,many consumers receive television programming-related content viabroadcast over a cable network to a television or similar display, withthe content often received via a set-top box (“STB”) from the cablenetwork that controls display of particular television (or “TV”)programs from among a large number of available television channels,while other consumers may similarly receive televisionprogramming-related content in other manners (e.g., via satellitetransmissions, broadcasts over airwaves, over packet-switched networks,etc.). In addition, enhanced television programming services andcapabilities are increasingly being provided to consumers, such as theability to receive television programming-related content that isdelivered “on demand” using Video on Demand (“VOD”) technologies (e.g.,based on a pay-per-view business model) and/or various interactive TVcapabilities. Consumers generally subscribe to services offered by acable network “head-end” or other similar content distribution facilityto obtain particular content, which in some situations may includeinteractive content and Internet content.

Consumers of content are also increasingly using a variety of devices torecord and control viewing of content, such as via digital videorecorders (“DVRs”) that can record television-related content for laterplayback and/or can temporarily store recent and current content toallow functionality such as pausing or rewinding live television. A DVRmay also be known as a personal video recorder (“PVR”), hard diskrecorder (“HDR”), personal video station (“PVS”), or a personaltelevision receiver (“PTR”). DVRs may in some situations be integratedinto a set-top box, such as with Digeo's MOXI™ device, while in othersituations may be a separate component connected to an STB and/ortelevision. In addition, electronic programming guides (“EPGs”) areoften made available to aid consumers in selecting a desired program tocurrently view and/or to schedule for delayed viewing. Using an EPG anda DVR, a consumer can cause a desired program to be recorded and canthen view the program at a more convenient time or location.

As another example, the Internet has become a frequently used source ofmultimedia content. Consumers may obtain music downloads, pictures,audio or video clips, graphical or textual information, or othermultimedia content by accessing suitable Web sites and downloading thedesired piece(s) of multimedia content from the accessed Web site(s).

However, as the multimedia industry grows and as additional types ofcontent are increasingly being stored and made available to consumers,it is becoming increasingly difficult for consumers to effectivelymanage the content and the related capabilities of the various devicesthat are available to present the content or to control the contentpresentation. Moreover, the increase in number and types of mediadevices within homes (e.g., televisions, handheld and desktop computingsystems, stereos, speakers, cellphones, DVRs, STBs, etc.) makes the taskof effectively presenting and otherwise managing the multimedia contenteven more difficult. For example, it is quite common for each consumer'shousehold to possess several remote controls, with each remote controlbeing operative to control only its corresponding media device. Thus,consumers are forced to familiarize themselves with the features of eachindividual remote control, which can become difficult or frustrating toa consumer as additional media devices (and their remote controls) areadded to a household or whenever such media devices are modified orupgraded to other models and versions. Moreover, in some situations,various electronic devices in the household may not have associatedremote controls or may otherwise require the consumer to activelyoperate their controls, which similarly can lead to difficulties andfrustrations for the user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an embodiment of a content management(“CM”) system environment in which an embodiment of a content managementsystem interacts with one or more other devices in the environment.

FIG. 2 is a block diagram of an embodiment of the CM system and one ofthe other devices in the environment shown in greater detail.

FIG. 3 is a block diagram illustrating an embodiment of the CM systemoperating in conjunction with one of the other devices and a mediaserver.

FIG. 4 is a block diagram illustrating an embodiment of the CM systemthat can be remotely accessed via a network.

FIG. 5 is a block diagram illustrating use of an embodiment of the CMsystem to control other types of devices in the CM environment.

FIG. 6 is a flowchart of an embodiment of a routine for operating anembodiment of the CM system.

FIG. 7 illustrates an example embodiment of a GUI for a contentmanagement system.

Non-limiting and non-exhaustive embodiments are described with referenceto the figures, wherein like reference numerals refer to like partsthroughout the various views unless otherwise specified.

DETAILED DESCRIPTION

Techniques are described below for managing various types of devices invarious ways, such as via a graphical user interface (“GUI”) of aContent Management (“CM”) system executing on a computing device. In atleast some embodiments, the CM system on the computing device interactswith various other content presentation control devices (e.g., STBs,DVRs, media center devices, etc.) and/or other media devices that store,present, control or otherwise manage content, and in some embodimentsmay interact with a variety of other types of electronic devices(whether instead of or in addition to content presentation controldevices and other media devices). In at least some such embodiments, theCM system acts as a “control point” for such other devices, such as bypassing control requests to those devices (e.g., in response to userselections or other actions via the GM system GUI) and/or by retrievingdata from those devices (e.g., for display in the CM system GUI to oneor more users). Moreover, in at least some embodiments the underlyingcontrol logic (including business rules) for the other devices,information about current state of the other devices and/or content usedby the other devices are maintained at the other devices, and if so mayor may not be available to the CM system or the computing device.

In at least some embodiments, at least some content being managed by theCM system via the other devices includes television programming-relatedcontent, such as with the CM system receiving and using EPG metadatainformation for the content (e.g., via interactions with an STB), whilein other embodiments a variety of other types of content may be managedvia the other devices. Moreover, control requests may take a variety offorms in various embodiments, and may vary based on a type of devicebeing managed and/or a type of content associated with the device, withexample control requests including requests to record a televisionprogram, to playback recorded content, to request EPG metadatainformation so that the CM system can present the EPG information in theGUI or otherwise use the information, to tune a media device to aparticular channel, and so forth. In addition, the CM system may furtherperform various other types of device interaction and content managementfunctions, as described in greater detail below.

In some embodiments, the CM system may control or otherwise communicatewith one or more media devices using different communication protocolsand/or interfaces. Non-limiting examples of such communication protocolsinclude Universal Plug And Play (“UPnP”), hypertext transfer protocol(“HTTP”), or other suitable protocol(s).

In some embodiments, the CM system may be used to control thepresentation of content by the media devices. For example, the CM systemmay instruct (or otherwise control) a media device to search for and/oraccess one or more content repositories for desired content or tootherwise retrieve content from one or more content sources, such asfrom a media server device via a local area network shared by the mediadevice and the CM system's computing device. After the desired contentis located from a suitable content repository, the CM system may controlthe manner in which the media device(s) accesses and presents thecontent, such as via one or more other presentation devices (e.g.,speakers, a stereo, a television display, a computing system display,etc.).

In some embodiments, the CM system may also be remotely used orotherwise remotely accessed by a user in various ways. Thus, whetheralternatively or additionally to physically using the CM system at thecomputing device, the user may remotely access the CM system in suchembodiments, such as via a network (e.g., the Internet), thereby givingthe user the capability to remotely manage media devices.

In some embodiments, the CM system may also interact with otherelectronic devices in a household, whether alternatively or additionallyto interacting with media devices. For instance, a “smart home” or“smart office” may be provided wherein the CM system controlselectromechanical devices that are present, such as a heating,ventilation and air conditioning (“HVAC”) system, lighting, householdappliances, or other types of devices. Thus, for example, if the userwishes to watch a recorded television program, the CM system may be usedto activate playback of the recorded television program by a DVR, whilethe CM system is also used to control the lighting, temperature, drapes,etc. in the household so as to give the user an optimum environment forviewing the television program, such as in a coordinated manner (e.g.,based on a single instruction from the user, such as from a previouslydefined association and/or previously defined preferences).

In some embodiments, the CM system may also provide a variety of othertypes of capabilities via a GUI to allow one or more users to manage,directly or through other devices, various types of multimedia content(such as television programming, photographs, music, video clips, audioclips, video games, etc.). Example embodiments of a CM system GUI andassociated content management capabilities are described in furtherdetail in U.S. patent application Ser. No. 11/121,313 entitled“Time-Based Graphical User Interface For Television ProgramInformation,” and in U.S. patent application Ser. No. 11/120,664entitled “Maintaining A Graphical User Interface State That Is Based OnA Selected Piece Of Content,” both filed on May 2, 2005, and assigned tothe same assignee as the present application, and which are each herebyincorporated by reference in their entirety.

For illustrative purposes, some embodiments are described below in whichspecific types of content are managed or otherwise controlled, such asdirectly by the CM system or indirectly through one or more electronicdevices, and in specific ways via specific example embodiments of the CMsystem. However, the techniques can be used in a wide variety of othersituations, and the specific illustrative details discussed herein arenot limiting. More generally, as used herein, multimedia “content”generally includes television programs, movies and other videoinformation (whether stored, such as in a file, or streamed), photos andother images, music and other audio information (whether stored orstreamed), presentations, video/teleconferences, videogames, InternetWeb pages and other data, and other similar video or audio content.

Referring first to FIG. 1, a network diagram illustrates an example useof an embodiment of the CM system in a home environment 395 forentertainment purposes, although the CM system could similarly be usedin business or other non-home environments and for purposes other thanentertainment. In this example, the home environment includes an STB orother media center 350 receiving external content 360 a that isavailable to one or more consumers (not shown) in the home environment395, such as television programming-related content 360 a or othermultimedia content for presentation on a television 370. Hereinafter,the media center 350 will be identified for illustrative purposes as oneexample type of media device that can be controlled by or that caninteract with the CM system, although other types of media devicesand/or electronic devices may also be shown and described in specificexamples.

Other types of audio and/or video content could similarly be received byand/or stored by the media center 350 and presented to the consumer(s)on the television and/or optional other content presentation devices 380(e.g., other televisions, a stereo receiver, stand-alone speakers, thedisplays of various types of computing systems, wireless devices, adigital picture frame, etc.) in the home environment 395.

The home environment 395 also includes an example computing system 300suitable for executing an embodiment of the CM system, as well as one ormore optional other local computing systems and/or storage devices 390with which the computing system 300 can interact via local network 385(e.g., a wireless or wired/cabled LAN). In the illustrated embodiment,the computing system includes a CPU 305 or other processor(s), variousI/O devices 310, storage 320, and memory 330, and the illustrated I/Odevices include a display 311, a network connection 312, acomputer-readable media drive 313, and other I/O devices 315 (e.g., akeyboard, a mouse, a pointing device, speakers, etc.). In someembodiments the computing system 300 may further interact with one ormore external computing systems, not shown (e.g., via the World Wide Webor otherwise via the Internet), such as to receive additional externalcontent 360 b and/or other information or services. In addition, in anembodiment that will be described below, the CM system can be accessedor otherwise operated remotely by a user via the Internet or othernetwork.

In the illustrated embodiment, the computing system 300 interacts withthe media center 350 in a variety of ways (e.g., via a directconnection, as part of local network 385, or as part of another network,not shown), including to receive information about current and/or futuretelevision programming-related content and/or other content 360 a. Inthis example, the information received by the computing system 300includes EPG metadata information for television programming-relatedcontent (e.g., from a cable network operator), and can further include avariety of other types of metadata for the content 360 a, whether fromthe cable network operator or other third-parties (e.g., from variousexternal computing systems, not shown, such as via external content 360b). This received content metadata is then stored in a content metadatadatabase 322 (or other suitable data structure) in the illustratedembodiment for use by the CM system.

An embodiment of the CM system 335 is executing in memory 330, and in atleast some embodiments comprises software or other machine-readableinstructions that can be executed by the CPU 305 or other processor. Oneexample of the CM system 335 would be a service application that runs inthe background (e.g., to programmatically receive and handle requests),while in another example would be an application program that runs inthe foreground and controls an appropriate display device (e.g., topresent a GUI of the CM system. The CM system may provide informationabout the available content to one or more users, such as via a GUI thatis displayed on the display device 311 and/or on one or more of thecontent presentation devices 370 or 380 or other computing systems 390(e.g., to one or more users in the home environment), and/or viainformation transmitted to remote users (not shown). For example, the CMsystem may provide some or all of the stored content metadata 322 to theusers, and also allow the users to manipulate the content in a varietyof ways (e.g., to select content for current presentation, for futurerecording, etc.), as well as in some embodiments maintaining a state ofthe GUI as displayed information changes. As the user providesinstructions related to manipulation of the content, those contentmanipulation instructions are in the illustrated embodiment stored in adatabase 324 (or other suitable data structure) on storage 320 for laterretrieval and use by the CM system, and at least some such contentmanipulation instructions may additionally in some embodiments beprovided to the media center 350 for use in actually performing thetasks to satisfy the manipulation instructions. In addition, in someembodiments the CM system may also interact with other executingprograms in order to provide additional information and/or functionalityto the user(s), such as one or more optional executing other programs339 in memory 330 or instead remote executing programs (not shown) onanother computing system.

In the illustrated embodiment, the CM system does not itself present theexternal content 360 a to the user(s), instead interacting with themedia center 350 to control the display of the content to the user viathe TV 370 and/or one or more of the other content presentation devices380. However, in other embodiments the CM system may instead receivesome or all of the external content 360 a, whether via the media center350 or instead directly from the source of that content, and if so coulddirectly control the presentation of that content to the user(s) on thedisplay device 311 and/or other content presentation device 370 or 380or other computing system 390. Conversely, in the illustrated embodimentthe computing system 300 and/or media center 350 may have access toadditional content, such as external content 360 b (e.g., from remotecomputing systems over the Internet), locally stored other content 360 con storage 320, and/or other locally accessible content (not shown),such as from one or more of the other computing systems/storage devices390—if so, the computing system 300 may in some embodiments directlypresent some or all of that content to the user(s), such as on displaydevice 311 (e.g., as part of the CM system GUI) and/or on one or moreother content presentation devices 370 or 380 or other computing system390, while in other embodiments may present some or all of that contentto the user(s) by instead interacting with the media center 350 tofacilitate the presentation (e.g., by providing the additional contentto be presented and/or associated instructions to the media center). Aspreviously noted, the additional content may be content related to theexternal content 360 a but from another source (e.g., additionaltelevision-related programming, such as streamed or downloaded over acomputer network) and/or multimedia content of other types (e.g., moviesand other video information, photos and other images, music and otheraudio information, presentations, video/teleconferences, videogames,Internet Web pages and other data, etc.).

The functionality of the CM system may be accessed in various ways indifferent embodiments. For example, some users may have physical accessto the computing system 300, and if so may interact with the various I/Odevices 310 to provide and/or receive information. Alternatively oradditionally, other users may use other client computing devices toremotely access the CM system, such as other local computing systems 390or instead remotely (e.g., via the World Wide Web or otherwise via theInternet, such as by using Web services or other remote accesscapabilities provided by the CM system). Such remote users may usesoftware or other functionality provided on the client computing systems(not shown), such as a browser, to interact with the CM system. In otherembodiments, users may receive functionality and/or information from theCM system indirectly via interactions with one or more other devices,such as the media center 350, which may directly receive thatfunctionality or information from the CM system before providing it tothe users.

For example, in some embodiments the computing system 300 and/or one ofmore of the computing systems 390 is a cellphone (e.g., a smartphone) orother mobile or handheld device with communications capabilities (e.g.,a Pocket PC computer, a PDA, an electronic organizer, etc.), and if sothe user may use that device to manage content in various ways. In suchembodiments, the communications between the cellphone or other deviceand other computing systems and devices in the example home environmentmay occur in various manners. For example, some devices may supportvarious types of local communication mechanisms (e.g., wirelesscommunications via Bluetooth, Wi-Fi, direct infrared, etc.) and/orcommunication mechanisms that may be used remotely (e.g., if the devicesupports Internet access, such as via GPRS and using HTTP or WAP, viaWeb services over HTTP, or instead using a standard telecommunicationslink for exchanging voice and other audio information), and may furthersupport various communication and messaging protocols (e.g., UPnP, HomeAudio Video Interoperability (or “HAVI”), etc.). When a device supportsboth local and remote communication mechanisms, the device may furtherbe configured to seamlessly switch between local and remotecommunications as appropriate, such as to automatically use localcommunications when they are available and to use remote communicationsotherwise, or instead may use various communications as directed by theuser. Remote communications may be appropriate, for example, when thedevice is remote from the home environment (e.g., outside a buildingthat contains the home environment, not shown, such as at significantdistances), such as to perform some or all data exchange via Internetaccess and/or using voice commands over a standard telecommunicationlink. In addition, various types of devices may have varyingcommunications abilities, and thus devices in various embodiments mayuse different forms of communication for a single type of informationexchange.

When a user uses a cellphone or other mobile or handheld device tomanage content, various types of information may be exchanged in variousways. For example, software to implement at least a portion of the CMsystem on the device and/or to interact with a remote executing CMsystem may be preinstalled on the device when provided to the user(e.g., by the distributor of the device) and/or may be downloaded to thedevice after the user has the device (e.g., as an executable file fromanother local computing system to which the device has a physical orwireless connection, or directly from a remote Web site if the devicesupports Internet access). EPG and/or other types of metadata maysimilarly be provided to the device in various ways, such as viaInternet access to a local computing system (e.g., computing system 300)and/or a remote computing system, or instead from a local computingsystem and/or media center device 350 via local communicationmechanisms. Commands and other information from the user may similarlybe provided from the device to computing systems and other devices inthe home environment in various ways, including via Internet accessand/or local communication mechanisms.

Computing system 300 is merely illustrative and is not intended to limitthe scope of the present invention. For example, the computing systemmay instead be comprised of multiple interacting computing systems ordevices, and may be connected to other devices that are not illustrated,including through one or more networks such as via the World Wide Web(“Web”) or otherwise via the Internet. More generally, the computingsystem 300 and/or the other computing systems 390 may each comprise anycombination of hardware or software that can perform the describedtechniques, including (without limitation) desktop or other computers,network devices, Internet appliances, PDAs, cordless and other wirelessphones, cellphones, devices with walkie-talkie and other push-to-talkcapabilities, personal/digital video recorders, pagers, electronicorganizers, television-based systems and various other consumer productsthat include appropriate inter-communication and computing capabilities.In addition, the functionality provided by the CM system may in someembodiments be distributed in additional systems, and the CM system mayhave multiple components that each provides a portion of thefunctionality of that system. Similarly, in some embodiments thefunctionality of some of the illustrated systems may not be providedand/or other additional functionality may be available.

While various items are illustrated as being stored in memory or onstorage while being used, these items or portions of them can betransferred between memory and other storage devices, such as forpurposes of memory management and data integrity. Alternatively oradditionally, in other embodiments some or all of the software system(or portions of it) may execute in memory on another device andcommunicate with the illustrated computing system via inter-computercommunication. Some or all of the system and/or data structures (e.g.,portions of a database, such as one or more tables or views or portionsthereof) used by the system may also be stored (e.g., as softwareinstructions or structured data) on a computer-readable medium, such asa hard disk, a memory, a computer network or other transmission medium,or a portable media article (e.g., a CD, DVD or flash memory device) tobe read by an appropriate drive or via an appropriate connection. Someor all of the system and data structures can also be transmitted viagenerated data signals (e.g., by being encoded in a carrier wave orotherwise included as part of an analog or digital propagated signal) ona variety of computer-readable transmission mediums, includingwireless-based and wired/cable-based mediums, and can take a variety offorms (e.g., as part of a single or multiplexed analog signal, or asmultiple discrete digital packets or frames). Such computer programproducts may also take other forms in other embodiments. Accordingly,one or more embodiments may be practiced with other computer systemconfigurations.

FIG. 2 shows an embodiment of the CM system 335 and media center 350 inmore detail, including illustrating interactions between the two. Forthe purposes of brevity and simplicity of explanation, only certainelements of the CM system 335 and/or the media center 350 are shown anddescribed herein.

In various embodiments, the media center 350 may comprise a DVR, STB,PVR, or other media device. The media center 350 may also be a“standalone” device, such as a Replay TV or TiVo device (as illustrativeexamples), or may be an integrated device that includes multiple typesof capabilities, such as Digeo's MOXI™ device or devices provided bymany multiple service operators (“MSOs”), as illustrative examples. Themedia center 350 may include one or more processors (not shown) and adata store 206. The data store 206 may store a variety of differenttypes of data, including but not limited to, external content 360 a,information received from the CM system 335 (including command andrequest information), state information, processing information, orother type of computer-readable information.

In the illustrated embodiment, the media center 350 includes one or moreservices 200, including a TV watching service and jukebox service in theexample of FIG. 2, that provide various functionalities and capabilitiesof the media center 350. These services 200 may also include acommunication service 202, such as a UPnP service and/or a HTTP servicein an example embodiment, which is used for communication with the CMsystem 335 and optionally other devices. For example, in someembodiments the communication service 202 may be implemented as a UPnPdaemon that runs in the background.

In the illustrated embodiment, the media center 350 exposes access toits services 200 via one or more application program interfaces (“APIs”)204—by exposing its services via the API 204, external systems (such asthe CM system 335) may communicate with the API 204 (e.g., via APIfunction calls) to access, use, or otherwise interact with thefunctionality provided by the services 200. Alternatively oradditionally, the CM system 335 may communicate directly with any of theservices 200 without communicating via the API 204, such as bytransmitting control instructions in a manner similar to a remotecontrol device (not shown) for the media center 350. In someembodiments, the CM system 335 communicates with the media center 350via the API 204 using one or more suitable protocols 208. Non-limitingexamples of a protocol 208 include UPnP, HTTP, a custom protocol, or anyother protocol that is suitable to allow two or more devices tocommunicate with one another, including higher-level protocols based onother communication protocols (e.g., by using Web service calls overHTTP). For the sake of completeness, a general description of the UPnPprotocol is provided next.

By way of general discussion, the UPnP architecture allows devices toconnect seamlessly and to simplify the implementation of networks inhome and corporate environments. The UPnP architecture offers pervasivepeer-to-peer network connectivity of PCs of all form factors,intelligent appliances, and/or wireless devices. The UPnP architectureis a distributed, open networking architecture that leverages TCP/IP andthe World Wide Web to enable seamless proximity networking in additionto control and data transfer among networked devices in the home,office, and everywhere in between.

The UPnP architecture enables data communication between any two devicesunder the command of a suitable control device on the network. The UPnParchitecture supports zero-configuration, networking, and automaticdiscovery for a breadth of device categories from a wide range ofvendors, whereby a device may dynamically join a network, obtain an IPaddress, announce its name, convey its capabilities upon request, andlearn about the presence and capabilities of other devices. UPnPnetworking involves the following steps:

1. Discovery: Given an IP address, the first step in UPnP networking isdiscovery. When a device (e.g., the media center 350, other contentpresentation devices 380, other computing systems 390, and so forth) isadded to the network, the UPnP discovery protocol allows that device toadvertise its services to control points on the network (e.g., the mediacenter 350 and/or the CM system 335 of the computing system 300).

2. Description: The next step in UPnP networking is description. After acontrol point has discovered a device, the control point may know verylittle about the device. For the control point to learn more about thedevice and its capabilities, or to interact with the device, the controlpoint retrieves the device's description from a uniform resource locator(“URL”), for example, provided by the device in the discovery message.The UPnP description for a device may be expressed in XML and mayinclude, for instance, manufacturer information like the model name andnumber, serial number, manufacturer name, URLs to vendor-specific websites, etc. The description may also include a list of any embeddeddevices or services (e.g., the services 200), as well as URLs forcontrol, eventing, and presentation. For each service, the descriptionmay include a list of the commands or actions that the service respondsto, and parameters or arguments for each action; the description for aservice 200 may also include a list of variables, which may model thestate of the service at run time, and which may be described in terms oftheir data type, range, and event characteristics.

3. Control: The next step in UPnP networking is control. After a controlpoint has retrieved a description of the device, the control point maysend actions to a device's service. One technique to perform this is fora control point to send a suitable control message to a control URL forthe service (provided in the device description). Control messages mayalso be expressed in XML using the Simple Object Access Protocol(“SOAP”) or other suitable format.

4. Event notification: The next step in UPnP networking is eventnotification, or “eventing”. A UPnP description for a service includes alist of actions that the service responds to and a list of variablesthat model the state of the service at run time. The service publishesupdates when these variables change, and a control point may subscribeto receive this information. The service publishes updates by sendingevent messages. Event messages contain the names of one of more statevariables and the current value of those variables.

5. Presentation: The next step in UPnP networking is presentation. If adevice has a URL for presentation, then the control point may retrieve apage from this URL, load the page into a Web browser, and depending onthe capabilities of the page, allow a user to control the device and/orview device status. The degree to which each of these may beaccomplished depends on the specific capabilities of the presentationpage and device.

Various additional details related to the UPnP protocol are available in“Understanding Universal Plug and Play: White Paper”, available athttp://www.upnp.org/download/UPNP_UnderstandingUPNP.doc, which is herebyincorporated herein by reference in its entirety.

While the UPnP protocol is described herein as one of the protocols 208that may be used on the local network 385 (which communicatively couplesthe computing system 300 to the media center 350 and/or to other devicesin the home environment 395, other types of protocols may be used, suchas HTTP, a custom protocol, or any suitable combination of protocols.

In the illustrated embodiment, the CM system 335 includes a GUI 210, acore module 212, a data store 214, a data manager 216, and one or moregateway modules 218. The GUI 210 may present EPG and/or other multimediametadata information, controls, or other tools and functionality toallow a user to view, arrange, manipulate, or to otherwise manage andcontrol content, media devices (e.g., the media center 350, the othercontent presentation devices 380, etc.), and/or other electronicdevices. An example of the GUI 210 is illustrated as GUI 100 in FIG. 7,and additional examples are disclosed in the other patent applicationsidentified above.

In the illustrated embodiment, the core module 212 serves as theunderlying control engine for the CM system, such as to generate orotherwise provide control requests and data requests (e.g., in responseto instructions from the GUI, such as based on user selections ofcontrols and other user actions). For example, the core module 212 maygenerate requests for EPG metadata, recording status information,multimedia content metadata, or other data related to the operation ofthe media center 350, and then provide the data to the GUI 210. Suchdata may be stored in a local data store 214 (e.g., at least a portionof the storage 320 of FIG. 1, such as that holds an SQL or otherrelational database) and/or other data repository or data structure,such as after it is retrieved or otherwise obtained from the mediacenter 350 or other source. In addition, in this illustrated embodiment,a data manager 216 cooperates with the core module 212 to manage theflow of data to, from, and within the CM system 335, such as to providean interface to the data store (e.g., by acting a database server for adatabase portion of the data store).

The control requests generated by the core module 212 may be sent to themedia center device 350 to control or otherwise manage the device, suchas requests to record a certain program at a certain time, to play backa recorded program, to turn ON/OFF, and so forth. The core module 212may also generate control and data requests directed towards otherdevices, such as the content presentation devices 380, other computingsystems 390, and/or any other device in the home environment 395 that iscommunicatively coupled to or otherwise associated with the CM system335. The core module 212 may also control or otherwise manage otheroperational features of the CM system 335, including communications,data storage/processing/management, and other operations.

In at least some embodiments, the CM system 335 physically or logicallycommunicates with the media center 350 via the local network 385, whichmay comprise an Ethernet network connection or other type of network(whether wired and/or wireless). In particular, at least some suchcommunications may include the CM system 335 invoking or otherwiseaccessing the API 204 of the media center 350 in order to control orotherwise manage one or more of the services 200 of the media center 350that are exposed via the API, such as via the UPnP Service 202 that ismade available via a network connection. In at least some embodiments,there may also be one or more other networks 385 a that the CM system335 uses to communicate with other devices, such as the devices 380,systems 390, and/or other devices and systems, such as by using one ormore communication protocols 208 a. The networks 385 a and the protocols208 a may be the same or different than the network 385 and the protocol208, respectively.

In at least some embodiments, the CM system 335 further includes one ormore gateway modules 218 to facilitate communication with other device.For example, a different gateway module 218 may be provided for eachcorresponding type of media center 350 or for other type of externaldevices/systems in communication with the CM system 335, such as to useAPIs provided by that type of device and/or to use one or morecommunication protocols supported by that type of device. Thus, eachgateway module 218 may communicate with one or more correspondingexternal devices using an appropriate protocol(s) 208 and/or 208 a, suchas by way of an interface 220 of that gateway module. For example, whenthe protocol 208 in use is UPnP, a gateway module 222 corresponding tothe media center 350 may include a UPnP interface 220 such that calls tothe media center's API 204 using the UPnP protocol are passed throughthe UPnP Service. Additional details regarding communicating between theCM system 335 and external devices will be discussed below.

In addition, in some embodiments the CM system 335 has an extensiblearchitecture that allows additional gateway modules to be added asappropriate. Thus, as additional media devices and/or other externaldevices are added to the home environment 395, one or more correspondingnew gateway modules 218 may be installed or modified in the CM system335. Moreover, certain gateway modules 218 may be deactivated or removedif their corresponding external device(s) is replaced, disconnected, orotherwise discontinued from use. The capability to add, remove, and/ormodify the gateway modules 218 provides a convenient and scalabletechnique to centrally manage or otherwise centrally control a pluralityof connected devices, without having to perform substantivemodifications to the underlying hardware and/or software of such devicesor of the CM system.

As described above, many media centers 350 provide an API 204 thatexposes services 200 of the media centers, thereby providing the CMsystem 335 with a mechanism to use to control or otherwise access theseservices 200 without having to perform hardware and/or softwaremodifications on the media centers 350. In some types of media centers350 and/or with other types of media devices, however, certainmodifications may be performed in at least some embodiments to improvethe interaction between the CM system 335 and the media center 350. Forexample, to implement a “silent operation” mode when the CM system 335is being used, the media center 350 of an embodiment may be designedand/or configured to be able to suppress on-screen displays or messagesthat typically arise in response to direct interactions with the mediacenter—if so, interactions initiated from the CM system 335 may insteadcause such displays or messages to be passed to the CM system 335 fordisplay within the GUI 210 or for other handling as appropriate.Similarly, other types of information that may be provided by the mediacenter in situations other than in response to direct interactions(e.g., messages related to conflicts that are detected, such as relatedto more programs to be simultaneously recorded and/or presented than canbe handled by the media center, and to resolution options for theconflicts) may instead be directed to the CM system 335 as part of sucha “silent operation” mode, such as for appropriate handling by the CMsystem 335 (e.g., by providing conflict resolution instructions from theCM system to the media center, such as in response to user instructionsvia the GUI 210).

As previously discussed, both the CM system 335 and the media center 350may be nodes on the same network in at least some embodiments, such asin the examples of FIGS. 1-2—such inter-connection allows for devicediscovery, command control, data retrieval, or other operationspertaining to the communication protocol used and/or pertaining to aspecific function being performed. In addition, the CM system 335 mayconnect to other external devices and/or content sources in variousways, including over the same network. Moreover, the CM system 335 mayobtain metadata (e.g., EPG information and/or metadata for other typesof multimedia content) and/or other information of interest fromconnected devices in various ways. For example, in a first scenario(“scenario 1”), the CM system 335 requests the metadata from the mediacenter 350, while in a second scenario (“scenario 2”) the CM system 335requests the metadata from an external source as part of externalcontent 360 b (although the CM system 335 may still retrieve other typesof information (e.g., scheduling data) from the media center 350 inscenario 2). In both scenarios 1 and 2, the protocol for the connectionbetween the CM system 335 and the media center 350 (and/or between theCM system 335 and some other media device) may be UPnP, UPnP and HTTP,only HTTP, some other protocol, a custom protocol, or any suitablecombination thereof. Examples of both of the connection scenarios areprovided below:

As an example involving connection scenario 1, the CM system 335 uses asuitable protocol (e.g., UPnP, HTTP, etc.) to communicate with the mediacenter 350 for device discovery, command interface, data retrieval, andother operations via the local network 385. The data retrieval mayinclude the EPG metadata information, which may be acquired by the mediacenter 350 in various ways (e.g., from a MSO head-end via a DOCSISmodem). The media center 350 makes the EPG data available to the CMsystem 335 as part of the data retrieval in the specified protocol.

As an example involving connection scenario 2, the media center 350 doesnot provide EPG or other multimedia content metadata to the CM system335, and thus the CM system 335 has a second connection to retrieve themetadata, such as involving a network connection to retrieve themetadata from an external data source (e.g., using XMLTV for EPGmetadata). The CM system 335 is also capable of aggregating data frommultiple sources, including data provided from media devices, Internetdata, and other data from various other data sources.

One example implementation of the media center 350 includes anintegrated media device that uses connection scenario 1 and the UPnPprotocol. The integrated media device uses a UPnP-based communicationservice (e.g., the communication service 202 on the media center 350)for communications between the CM system 335 and the API 204 using theUPnP protocol. The specific functions that are exposed through the UPnPcommunication service may vary from one media device to another based onwhat is available in the underlying API 204 and on the specificimplementation of the UPnP service. In the CM system 335, the UPnPinterface 220 of the gateway module 222 translates command and datarequests from the core module 212 into the UPnP protocol, and thenpasses the requests to the UPnP communication service 202 on the mediacenter 350. The UPnP interface 220 also passes data received from themedia center 350 via the UPnP service 202 to the core module 212 in anappropriate format. The data retrieval from the media center 350 mayincludes EPG metadata or other metadata.

Another example implementation of the media center 350 includes astandalone media device that uses connection scenario 2, with a firstconnection between the media center 350 and the CM system 335 (e.g.,using UPnP) and with a second connection between the CM system 335 andanother data source for external content 360 b (e.g., using HTTPprotocol). In this example, the first connection to the media center 350may, for example, use the UPnP protocol for device discovery and theHTTP protocol for the command interface and data retrieval (ofscheduling data), while the second connection to the external datasource may, for example, obtain EPG and/or other metadata using the HTTPprotocol for data retrieval. To enable such communications, one of theother gateway modules 218 of the CM system may contain both UPnP andHTTP interfaces 220, or the CM system may instead use two gatewaymodules that each support one of the UPnP and HTTP communicationprotocols, with the one or more gateway modules acting in a mannerpreviously described. In addition, a URL or other network addressinformation for an external source of EGP and/or other information maybe obtained by the CM system in various, such as based on a predefinedconfiguration for the CM system and/or dynamically from the media center350.

In the above examples, the external content 360 b has been described asoptionally including EPG metadata and/or other multimedia contentmetadata. In other embodiments, the external content 360 b may alsoinclude various other types of content, such as promotional types ofmaterials for users (e.g., a trailer for a movie, extra content such asdeleted scenes, etc.). For example, the GUI 240 may provide capabilitiesto allow the user to request some promotional materials and/or theability to select additional services (e.g., providing additionalcontent), such as for a fee.

FIG. 3 is a block diagram illustrating an embodiment of the CM system335 operating in conjunction with the media center 350 (and/or othermedia device) and one or more other media servers 302. In thisembodiment, the CM system 335 may instruct the media center 350 tosearch for or otherwise obtain media content of interest from one ormore of the media servers, such as by providing indications of thecontent of interest and/or of one or more of the media servers. Inresponse, the media center 350 may contact one or more suitable mediaservers 302 (e.g., media servers indicated by the CM system), obtainindications of pieces of media content available from the mediaserver(s) (e.g., a playlist or other multimedia content listing 304),and provide indications of at least some of the content (e.g., theplaylist or other content listing 304) to the CM system 335, such as fordisplay in the GUI 210. For example, if the CM system requestedinformation about specified pieces of media or types of media content,the media center may in some embodiments provide only indications ofmatching pieces of media (if any), while in the other embodiments mayprovide all of the identified media piece indications for analysis bythe CM system. In addition, if a media server does not provide aplaylist or other multimedia content listing, the media center may insome embodiments generate such a content listing after obtaining theindications of the available media pieces from the media server, and/orthe CM system may create one or more such playlists (e.g., if theinformation received from the media center does not include thoseplaylists). A content listing may include various metadata associatedwith the pieces of media content, such as title, author, creation date,file size, media type, format, resolution, and/or other characteristics.

After the CM system receives indications of available media from themedia center, the CM system may specify to the media center to play orotherwise present one or more of the media pieces, such as based onselections made by a user based on a playlist or other informationpresented in the GUI, with the media center performing the indicatedoperation (e.g., by presenting a piece of media content on an associatedpresentation device, not shown, such as after obtaining the media piecefrom an appropriate media server, or instead by directing an appropriatemedia server to cause the media piece to be presented, whether itself orvia an associated presentation device). In addition, the media centermay in some embodiments retrieve and store pieces of media content fromone or more such media servers, such as after receiving an instructionto present those media content pieces or instead when initiallygathering information about media content pieces that are available frommedia servers.

In the example embodiment of FIG. 3, the media server device(s) 302 maycomprise any type of suitable data repository that may store mediacontent of possible interest (e.g., photographs, audio files (such asMP3 files), video clips, movies, songs, Internet Web page content,graphics and text, video games, advertisements, or other types ofmultimedia content), including devices such as one or more hard diskdrives of a computer, servers, databases, mass storage units, tapedecks, CDs, DVDs, floppy disks, and/or other type of persistent ornon-persistent storage unit. In some embodiments, the media server(s)302 may be located in the same household or premises as the othercomponents of the home environment 395, while in other embodiments themedia server(s) 302 may be located remotely from the home environment395 and be accessible via a network such as the Internet, whetherinstead of or in addition to one or more local media servers. The mediaserver(s) 302 may be accessible by the media center 350 by way of anetwork 306, such as an internal network (e.g., an Ethernet or LANconnection) and/or an external network (e.g., the Internet), and mayfurther comprise wireless and/or wired/cabled connections. The mediaserver(s) 302 may communicate with the media server(s) 302 using one ormore suitable communication protocols, such as the communicationprotocols described herein or other protocols or combinations thereof.

In some embodiments, the media server(s) 302 may further becommunicatively coupled or otherwise accessible to the computing system300, whether instead of or in addition to a connection 306 to the mediacenter. In such embodiments, the CM system 335 may thus communicatedirectly with the media server(s) 302 without the media center 350, suchas via a network 307 between the CM system 335 and the media server(s)302 that uses one or more suitable protocols. As with the network 306,the network 307 may comprise an internal network (e.g., an Ethernet orLAN connection) or an external network (e.g., the Internet), and mayfurther comprise wireless and/or hardwire connections.

In at least some embodiments, a user may use the GUI 210 to instruct theCM system 335 to search a particular media server 302 and/or to searchfor a particular piece of media content. For example, a search field inthe GUI 210 may allow a user to create a customized search for musichaving a specific genre. The core module 212 of the CM system 335 thenpasses this search request to an appropriate gateway module 218, whichtransmits the request to the media center 350 using the local network385 and a suitable protocol 208.

The media center 350 then performs a search of media server(s) based onthe criteria provided by the user. Various techniques may be used by themedia center 350 to perform a search. In one example implementation, themedia center 350 may poll each media server 302 via the network 306 todetermine whether each media server 302 contains pieces of media orother content that match the search criteria. In such an implementation,each media server 302 may return a playlist 304 that lists pieces ofmedia that fit the search criteria. The playlist 304 is then provided bythe media center 350 to the CM system 335.

In another example implementation, the media server(s) 302 may return aplaylist 304 or other content listing that contains indications of allof their available stored media. Thereafter, the media center 350 mayparse or otherwise process the playlist 304 to obtain a resulting listof only media that fit the search criteria. The resulting list is thenprovided by the media center 350 to the CM system 335. In yet anotherembodiment, the media center 350 may provide the playlist 304 of allavailable stored media to the CM system 335, and the CM system 335 mayprocess the playlist 304 so that only the pieces of media that fit thesearch criteria are displayed on the GUI 210. The contents of theplaylist 304, whether listing all of the pieces of media in the mediaserver(s) 302 or a subset thereof, may be presented on the GUI 210and/or on a display coupled to the media center 350 (such as on a screenof the television 370 or on a display of the media center 350). Inanother embodiment, the CM system 335 may communicate directly with themedia server(s) 302 via the network 307 to obtain a playlist 308 ofavailable media at the media server(s) 302. The playlist 308 maycomprise a list of all available media or a list of available mediahaving certain criteria, such as user-defined criteria. The contents ofthe playlist 308 may then be displayed by the core module 212 on the GUI210 of the CM system 210.

Thereafter, the user may operate the GUI 210 to select a piece of mediacontent from the playlist 304 or 308 so as to cause the selected pieceof media content to be played back or otherwise presented. For example,if the user selects a particular song for playback from the playlist 304or 308 using the GUI 201 (including perhaps specifying a particularcontent presentation device 380 to play back the song), the CM system335 communicates this request to the media center 350. The media center350 receives this request from the CM system 335 via the network 385,obtains a copy of the requested song from the appropriate media server302, and sends the copy of the song to the specified contentpresentation device 380 for playback. Alternatively, the media center350 may request the media server 302 to provide the copy of the songdirectly to the specified content presentation device 380 for playback.

In addition, in at least some embodiments the CM system 335 may beremotely accessed by a user, such as via the Internet or other network.This remote access capability allows the user to use the CM system 335to remotely control content presentation control devices accessible tothe CM system (e.g., to program the media center 350 to record aprogram, to view and manipulate EPG or other metadata, to turn a homeappliance ON or OFF, to activate or deactivate parental controlsettings, and various other functions that may be performed by using theCM system 335 to control the media center 350, content presentationdevices 380, and/or other computing devices 390), without necessarilybeing present in the household environment 395 and physically using thecomputing device 300 that provides the CM system 335. Accordingly, FIG.4 is a block diagram illustrating an embodiment of the CM system 335that may be remotely accessed via a network.

The illustrated embodiment of the CM system 335 in FIG. 4 includes aremoting façade 400 or other suitable component or layer to supporttwo-way communication. While the remoting façade 400 may in someembodiments merely provide access to other elements 212-222 of the CMsystem such as were described with respect to FIG. 3, in the illustratedembodiment the remoting façade provides a higher-level interface tothose CM system elements that is better suited for communications 402with other elements of the computing device 300. In some embodiments,the communications 402 are performed using Microsoft's .Net remoting,with the remoting façade 400 being a .Net object that is exposed toother processes and/or components using the .Net remoting communications402, while in other embodiments a variety of other types of remoteprocedure call (“RPC”) or similar intercommunication mechanisms mayinstead be used. In addition, the illustrated embodiment of thecomputing device 300 includes a Web server 404, which may be separatefrom or part of the CM system 335. The Web server 404 includes a Webservice 414 that exposes the remoting façade 400 to a network 416 (e.g.,a wireless and/or wired/cabled hardwire network, such as the Internet, aWAN, a LAN, a telecommunications company network, and/or other type ofnetwork).

The illustrated Web server 404 further includes a Web application 406,which in at least some embodiments comprises an Active Server Page(“ASP”) ASP.Net Web application that provides a Web GUI front-end forthe CM system 335. The Web GUI may be provided alternatively oradditionally to the GUI 210 of FIG. 3, such as to provide remote userson the network 416 with capability to access and use the CM system 335via the Web GUI while local users access and use the CM system 335 viaGUI 210. The Web application 406 in the illustrated embodiment includesCascade Style Sheets (CSS) 408 that may be used to provide flexibilityand ease in generating and changing the look-and-feel of the Web GUI,ASPX pages 410 to support its operation, and code 412 to providecommunication between the Web application 406 and the remoting façade400 of the CM system 335 via .Net remoting communication 402. In otherembodiments, a separate Web GUI may instead not be provided, such as ifremote users receive the same GUI 210 as would local users. In addition,in some embodiments the Web GUI may be tailored or customized to supportvarious types of devices (e.g., by varying formatting and/or amount ofdata provided), such as devices with limited display area, connectionspeed and/or processing power (e.g., cellphones and other handheld ormobile devices, such as smartphones).

The Web GUI and associated functionality provided by the Web application406 and the Web service 414 are provided by the network 416 to one ormore remote devices 418 in the illustrated embodiment. A suitablecommunication protocol may be used on the network 416 to allowcommunication between these elements and the remote device(s) 418,including (without limitation) HTTP, HTTPS, SOAP, and/or othercommunication protocols. Examples of a remote device 418 may includevarious types of mobile devices (e.g., a laptop, PDA, cellulartelephone, Blackberry, GPS device, or other mobile wireless device) aswell as stationary computing devices (e.g., a personal computer (PC),workstation, and so forth).

The illustrated embodiment of the remote device 418 includes a Webbrowser 420 or other type of browser or application that is suitable tocommunicate with the Web server 404 and/or render the Web GUI providedby the Web application 406. For instance, the user may use the Webbrowser 420 to access an interactive Web page and/or Web site where theWeb GUI is provided and accessible to allow the user to enter (or view)commands and data, or instead may use a CM application 422 tocommunicate with a Web service 414 provided by the Web server 404 usinga suitable protocol (e.g., HTTP and SOAP). While the illustratedembodiment of the remote device includes both a Web browser 420 and a CMapplication 422 (which in the illustrated embodiment includes a Webservice gateway module 424, such as a plug-in module for the CMapplication), in other embodiments a remote device may include only onesuch access mechanism. In this example embodiment, the user may generatecontrol requests and data requests using the Web GUI rendered by the Webbrowser 420, with such requests then communicated by the Web servicegateway module 424 to the Web service 414, which in turn communicatesthe requests to the remoting façade 400 so that the appropriateoperations may be performed by the elements 212-222 of the CM system 335to fulfill the requests. In addition, while not illustrated here, inother embodiments the Web browser 420 may execute a Web page from Webapplication 406 containing an embedded application (e.g., using ActiveX,s Java applet, client-side JavaScript, etc.) that provides at least aportion of the CM system functionality and that can interact with Webservice 414.

While various embodiments of the CM system 335 have been described inthe context of controlling media devices (such as the media center 350)in connection with media presentation, in at least some embodiments useof the CM system 335 may be extended to operations and/or devices thatare alternative or additional to media devices. One example of suchadditional use of the CM system 335 is in connection with homeautomation or a “smart home.” For example, in a “smart home” embodimentillustrated with respect to FIG. 5, a home environment 395 includes oneor more other electromechanical devices that are coupled or otherwiseaccessible to the CM system 335 and that may be controlled by the CMsystem. In particular, a plurality of electromechanical devices 500 ofvarious types are present in the home environment in the illustratedembodiment, with examples of the electromechanical (E/M) devicesincluding lighting systems, HVAC systems, security systems, automateddrapes, automated wall panels that may be raised/lowered orretracted/extended, telephones, appliances, and various other E/Mdevices.

Some E/M devices 500 may have their own associated processor(s) and/orcommunication capability (e.g., “intelligent” E/M devices), and may benetworked to the media center 350, to the other computing systems 390,and/or to the CM system 335 via respective communication networks 504,502, and 506. The communication networks 504, 502, and 506 may comprisenetworks similar to the networks 385 and 385 a described previouslyabove and may carry communications based on one or more suitableprotocols (e.g., UPnP, HAVI, HTTP, etc.). With intelligent E/M devices500, the CM system 335 may directly communicate control requests anddata requests directly to such devices via the network 506, withoutnecessarily having to communicate through some other intelligent devicein order to initiate an action.

With less intelligent E/M devices 500, an intermediate controllingdevice is used as an interface between the CM system 335 and such E/Mdevice(s). For example, automated drapes may not have their ownassociated processor and/or communication interface. Rather, theseautomated drapes may have an electromechanical actuator that may becontrolled by the computing system 390 or by some other intelligentcontrolling mechanism, such as via a connection provided by the network502. Therefore, for such less intelligent E/M devices 500, the CM system335 may communicate with the computing system 390 via the network 385 aand the protocol 208 a, with the computing system 390 in turninteracting with the appropriate E/M devices via the network 502.

In some embodiments, operation of at least some of the E/M devices 500may be correlated or otherwise tied to media presentation. For instance,if the user wishes to watch a movie, various other devices may becontrolled in a coordinated manner to facilitate that media presentation(e.g., to set a certain room temperature, lighting level, deactivationof a telephone by directing incoming calls to voicemail, etc.).Moreover, in some situations a user may further desire to start cookingpopcorn or a pizza in the oven. Therefore, the media center 350 (whichinitiates playback of the movie) under control by the CM system 335 maycommunicate the playback-related information and/or commands (e.g., dateand time at which playback will occur, requests to perform some action,or other commands or information) to the appropriate E/M devices 500and/or to the computing systems 390, or the CM system 335 may insteaddirectly interact with those E/M devices 500 to effect the desiredresults. The appropriate E/M devices 500 will then respond as directed,such as to reduce the lighting levels, deactivate the telephone, begincooking popcorn, and so forth.

Thus in the embodiment described above, context information may bemaintained across these multiple types of E/M devices 500, the mediacenter 350, the other computing systems 390, and/or the CM system 335.Maintaining such context insures, for instance, that certain devices arenot activated or deactivated when they should not be. For example, bymaintaining context (e.g., knowing that the media center 350 ispresently controlling presentation of a recorded movie), aninappropriate action is not taken (e.g., opening all the drapes andwindows to let excessive amounts of light and noise into the homeenvironment 395).

Moreover in some embodiments resources may be shared between the variousdevices of FIG. 5. For example, state information may be shared betweenthe various devices so that they know the state of each other, thusallowing devices to optimize or time their operation based on the statusof the other devices. As another example, certain user interfaces, datarepositories, communication interfaces, networks, or other elements maybe shared by the various devices, so that each device need notnecessarily provide their own of these elements. Data re-use and datasharing may be implemented to reduce instance of repetitive datageneration, memory accesses, or other redundancy.

FIG. 6 is a flowchart 600 of an embodiment of a routine for operatingthe CM system 335. The routine begins at step 602, where connection withone or more external devices is established based on one or morecommunication protocols. For instance, at least some such other devicesmay be previously known to the CM system (e.g., based on priorinteractions), while other devices may be dynamically identified (e.g.,via UPnP discovery, description, etc. steps as described previouslyabove).

At step 604, the CM system 335 next sends a data request to a connecteddevice (e.g., based on a connection and protocol established in step 602for that device), such as to a media device (e.g., the media center350), other computing system(s) 390 and/or one or more external datasources. The data request may be sent in response to user input via aGUI displayed to a user (not shown) or independently by the CM system335 (e.g., as part of a startup routine to obtain metadata for use withsuch a GUI). At step 606, the CM system receives the requested data,which may then be presented on a GUI at step 608 (e.g., for viewingand/or manipulation by the user).

In the illustrated embodiment, the routine next continues to step 610 tosend a control request to a connected device (e.g., based on a userselection via the GUI after the user has seen the requested data),although in other embodiments the routine may continue instead to step616. For instance, the user may have reviewed a returned listing ofrecorded media and issued a control request for a selected one of thelisted pieces of media to be played back. The connected device theninitiates performance of the requested operation(s) based on the controlrequest, such as is illustrated here at step 612 for the sake ofconvenience, which may include interactions with one or more otherpresentation devices. For instance, if the control request is a requestto play back a piece of recorded media, the media center 350 may providethe recorded media to the television 370 for presentation thereon.Alternatively or additionally, the connected device may perform therequested operation directly. For instance, if the connected device isan intelligent HVAC system and the control request is a request toreduce room temperature, the HVAC system may directly change athermostat.

At a step 614, the CM system 335 optionally receives status data orother feedback from the connected device based on its performance of thecontrol request, with the status data then presented to the user in theGUI at step 608. For instance, the status data may indicate that atelevision program is being recorded, playback is completed,insufficient memory space is available, the connected device has beenturned ON or OFF, and so forth. At step 616, the CM system 335 thenmonitors for additional or subsequent control and/or data requests, suchas from the GUI and/or from remote users via programmatic requests(e.g., via Web service 414). The CM system continues to monitor forthese requests until a request is detected at step 618, with the routinethen returning to steps 604 or 610 respectively to perform correspondingdata or control requests.

In some embodiments, the functionality provided by the routine discussedabove may be provided in alternative ways, such as being split amongmore routines. Similarly, in some embodiments the illustrated routinemay provide more or less functionality than is described, such as whenthe amount of functionality that is provided is altered. In addition,while various operations may be illustrated as being performed in aparticular manner (e.g., in serial or in parallel) and/or in aparticular order, in other embodiments, the operations may be performedin other orders and in other manners. In addition, in some embodimentscertain operations may be embodied in software or othercomputer-readable instructions stored on a computer-readable medium,such as software stored in the memory 330 of the CM system 335 of FIG.1.

FIG. 7 illustrates an example of information and capabilities providedby an embodiment of the CM system, and in particular illustrates variousfeatures of an example GUI, including a variety of information anduser-selectable controls. While specific layouts and formats are shownin FIG. 7, such specific details are not intended to be limiting orexhaustive. Similarly, while specify types of devices are illustrated asbeing currently controlled or otherwise managed via the GUI, a varietyof other types of content presentation control devices and other mediadevices, as well as other electronic devices, may similarly beassociated with and managed via the GUI in other embodiments.

The example GUI 100 embodiment illustrated in FIG. 7 is displayingprogram listing information for live TV in a view area 140, with thedisplayed listings initially based on a current time 195. The TVlistings in the view area 140 are initially shown in a grid format, withcolumns of the grid corresponding to successive time units of thirtyminutes each and with each row corresponding to a different TV channelthat may show one or more TV programs during the displayed time period.The GUI also includes a variety of other areas, including a menu bar 105with dropdown menus, primary content-type navigation tabs 110, secondarycontent-type-specific navigation tabs 115, a view control area 125 thatincludes view and filter controls 120 and time controls 130, a detailarea 150, and a status control sidebar area 170 that includes virtualremote control functionality 160 and a user-focused content summary area165. While not illustrated here, in some embodiments the GUI may furtherhave a status bar display area along the bottom of the GUI below thedetail area and status control sidebar area.

The view control area 125 also includes a first timebar that correspondsto a relatively long first time period at a first time scale (which isselected based in part using time controls 130) and that may in certainsituations display limited information about content (although in thisexample the first timebar does not display any limited information aboutany television programs or other content), while the view area 140illustrates detailed information about content for a relatively shortsecond time period at a second time scale, with a second timebarrendered at the top of the view area 140.

The example GUI 100 also illustrates a variety of controls andinformation 505-545 in the status control sidebar area 170, such as toprovide virtual remote control capabilities in order to controlrecording of content on multiple devices (e.g., multiple devices withDVR capabilities) and/or presentation of content on multiple devices(e.g., multiple televisions). For example, the illustrated sidebar areacontains multiple user-selectable device control tabs 505, with each tabrepresenting a device that can be controlled, such as with respect topresenting content and/or recording or otherwise obtaining content forlater presentation. In this example, two television devices areavailable to which live or previously recorded content can be directed,with the “Main TV” device currently selected as indicated by tab 505 a.The current content information section 520 indicates that “TheSimpsons” television programming is currently being directed to the“Main TV” television device for presentation, while other content (notshown) may be being directed to the “Bedroom” television device. In someembodiments, controls may be sent directly to the device beingcontrolled (e.g., by directing the “Main TV” television device to changechannels), while in other embodiments one or more intermediate devicesmay be involved in the controls (e.g., by directing an STB, not shown,to control live television content sent to the currently selectedtelevision device and/or by directing a DVR, not shown, to directrecorded television content sent to the currently selected televisiondevice). Moreover, in some embodiments one or more such intermediatedevices may each be able to interact with and/or control multiplepresentation devices, such as to have a single STB and/or DVR controlcontent being provided to two television devices (e.g., by using a MOXIdevice that provides such functionality).

Content to be directed to a currently selected device may be selected invarious ways. For example, user-selectable channel controls 540 areshown for manipulating a current channel, and in other embodiments othercontrols (e.g., user-selectable numeric buttons, not shown) may beavailable for specifying a particular channel. In addition, in thecurrent embodiment various program listing information for live TV isshown in a separate view area in grid format, and additional informationis displayed in the detail area for a “Who Wants to Be a Millionaire”television program that is currently selected in the view area—in thisembodiment, a user-selectable “Watch Now” control in the detail area maybe selected to cause the currently selected program to be directed tothe currently selected television device, and similar functionality maybe available for some or all of the television programs indicated in theview area (e.g., via a pop-up menu). The illustrated embodiment of thesidebar also includes a user-focused content summary area 515 related toupcoming recordings and previously recorded programs, and while adisplay of previously recorded programs in section 515 is notillustrated here, a displayed previously recorded program may beselected for current presentation on the currently selected televisiondevice (e.g., via instructions sent to a DVR or other storage device onwhich that program is stored).

A playlist section 510 is also shown in the illustrated embodiment,although no content is displayed in this example for a current playlist.In at least some embodiments, content of various types may be selectedfor inclusion in one or more such playlists, and content displayed in aplaylist in section 510 may similarly be selected for currentpresentation on the currently selected television device (e.g., viainstructions sent to a DVR or other storage device on which that programis stored). In the illustrated embodiment, content in playlists mayinclude various forms of content, such as video information (e.g.,television programming), digital music, photos and other images, etc. Insome embodiments, the device that is currently selected for control mayaffect the information in the playlist section in various ways, such asto select a current playlist based on a current device being controlled(e.g., based on the type of device, such as to select a playlist havingvideo information when the device being controlled is a televisiondevice) and/or to enable content for current selection only ifappropriate for the currently selected device (e.g., to disable video orimage information if the currently selected device is a speaker, but toallow video, images and music to be presented on at least some types oftelevision devices, such as based on capabilities of the device).

The illustrated embodiment of the sidebar control area also includes avariety of other controls related to content being presented and toother types of user instructions that may be provided. For example,various user-selectable controls may affect the type of content beingpresented from among various sources, such as live televisionprogramming (e.g., from a cable company or satellite provider), a menuor other functionality from a DVR (e.g., a MOXI device), and a ticker onat least a portion of the display of the currently selected televisiondevice (e.g., with headline news, such as from a cable company or from aprovider associated with the DVR). Various interactivity controls 530may further allow the user to make various selections, such as from amenu or from among multiple options. Various controls 535 may also beused to control the presentation of content, such as to pausepresentation, skip backward, skip forward (when available, such as withother than live content) and rewind or fast forward, as well as toprovide an instruction to record content that is currently beingpresented. One or more specialized controls 545 may also be provided,such as to correspond to functionality that may be available on onlycertain devices, such as on certain types of STBs and/or DVRs.

While various exemplary details have been provided, the describedtechniques may be used in a variety of other ways. In addition, thedescribed techniques are applicable to architectures other than aset-top box architecture or architectures based upon the MOXI™ system,for instance. For example, an equivalent system and applications may bedeveloped for other DVRs and STBs. The methods and systems discussedherein are applicable to differing protocols, communication media(optical, wireless, cable, etc.) and devices (such as wireless handsets,electronic organizers, personal digital assistants, portable emailmachines, game machines, pagers, navigation devices such as GPSreceivers, etc.), and other devices are able to be controlled or tocontrol other devices in the manner described herein. In addition, datastructures discussed above may be structured in different manners, suchas by having a single data structure split into multiple data structuresor by having multiple data structures consolidated into a single datastructure. Similarly, in some embodiments illustrated data structuresmay store more or less information than is described, such as when otherillustrated data structures instead lack or include such informationrespectively, or when the amount or types of information that is storedis altered.

Several specific protocols have been identified and described herein forproviding communication capabilities between different devices. Variousother mechanisms may be used to allow computers (or other devices) andexecuting software applications to interact. For example, remoteprocedure call (“RPC”) protocols allow a program on one computer tocause a program on another computer to be executed, and variousobject-oriented and other architectures such as CORBA (“Common ObjectRequest Broker Architecture”), DCOM (“Distributed Component ObjectModel”), Java RMI (“Remote Method Invocation”), JavaSpaces, Jini, andJXTA provide similar capabilities. In addition, a variety of middlewareprograms may be implemented to connect separate applications (often ofdistinct types and from unrelated sources) to allow communication. Forexample, various EDI (“Electronic Data Interchange”) networks may beused to provide mechanisms to allow a computer system of one user of thenetwork to send data to a computer system of another user of thenetwork.

The widespread popularity of the World Wide Web (“Web”) has providedadditional opportunities for computers to inter-communicate. Forexample, much current Web use involves users interactively requestingWeb pages from Web servers (e.g., via executing Web browser applicationsof the users) and receiving the requested information in response. Inaddition to such interactive user specification of requestedinformation, there is also growing use of the Web to support theprogrammatic interaction of remote applications to exchange informationvia defined APIs, such as via Web services. Web services, such as thosedescribed herein, allow heterogeneous applications and computers tointeract, and may be defined and implemented using a variety ofunderlying protocols and techniques. For example, some Web serviceimplementations return data in computer-readable XML format using HTTPin response to a Web service invocation request specified as a URI(“Uniform Resource Identifier”), such as a URL that includes a specifiedoperation and one or more query parameters. In other implementations,additional underlying protocols are used for various purposes, such asSOAP for standard message exchange, WSDL (“Web Services DescriptionLanguage”) for description of service invocation interfaces (e.g., usingXML format), and UDDI (“Universal Description, Discovery, andIntegration service”) for discovery of available services. These are buta few examples that may be used in various embodiments.

In the description, numerous specific details have been given to providea thorough understanding of embodiments. The embodiments may bepracticed without one or more of the specific details, or with othermethods, components, materials, etc. In other instances, well-knownstructures, materials, or operations are not shown or described indetail to avoid obscuring aspects of the embodiments. Thus, althoughspecific embodiments have been described herein for purposes ofillustration, various modifications may be made without deviating fromthe spirit and scope of the invention. Accordingly, the invention is notlimited except as by the appended claims and the elements recitedtherein. In addition, while certain aspects of the invention arepresented below in certain claim forms, the inventors contemplate thevarious aspects of the invention in any available claim form. Forexample, while only some aspects of the invention may currently berecited as being embodied in a computer-readable medium, other aspectsmay likewise be so embodied.

1. A method for enabling a computing device to be remotely accessed tocontrol media device and a heating, ventilation, and air conditioningsystem, the method comprising: configuring the computing device as acentral control point, the computing device capable to generate controlrequests and data requests related to operation of the media device andthe heating, ventilation, and air conditioning system using a coremodule of a management system of the computing device; separating thecore module of the management system from a remote graphical userinterface of the management system, by using a Web application of a Webserver to generate the remote graphical user interface and to make theremote graphical user interface available to a browser of a remotedevice via a network to allow the browser to render the remote graphicaluser interface to provide access and use of the management system to aremote user; providing a Web service of the Web server to allow theremote device to communicate with the core module; providing a remotinglayer to allow the Web application and the Web service to communicatewith the core module; providing a local graphical user interface forsimultaneous access and use of the management system by a local user;suppressing on-screen display of messages on at least one of the mediadevice or the heating, ventilation, and air conditioning system that isaccessed by the remote user; receiving a message from at least one ofthe media device or the heating, ventilation, and air conditioningsystem accessed by the remote user; and displaying the message via theremote graphical user interface in lieu of the suppressed on-screendisplay of the messages.
 2. The method of claim 1 wherein the graphicaluser interface is usable by the remote device to instruct the coremodule to generate the control requests and the data requests.
 3. Themethod of claim 1 wherein the method further comprises: facilitatingcommunication between the core module and the media device and theheating, ventilation, and air conditioning system via one or moregateway modules, wherein each of the one or more gateway modules isassociated with the media device or the heating, ventilation, and airconditioning system.
 4. The method of claim 1 wherein the method furthercomprises: receiving via the Web service a user command entered via thegraphical user interface; and transmitting via the remoting layer theuser command to the core module.
 5. The method of claim 1 wherein theWeb application displays via the graphical user interface informationgenerated by the core module in association with at least one of thecontrol requests or data requests.
 6. A non-transitory computer-readablemedium having stored thereon instructions for enabling a computingdevice that can control media device and a heating, ventilation, and airconditioning system to be remotely accessed, by: using a computingdevice as a central control point to provide control requests and datarequests related to operation of the media device and the heating,ventilation, and air conditioning system, the computing device includinga management system having a core module to generate the controlrequests and data requests to the media device and the heating,ventilation, and air conditioning system; separating the core module ofthe management system from a remote graphical user interface that can beused to interact with the management system, wherein the remotegraphical user interface is provided by Web application and is madeavailable by the Web application to a remote device via a network, andwherein a remoting layer being is used to allow the core module and theWeb application to communicate with each other, and wherein a Webservice being is used to allow the remote device to communicate with thecore module through the remoting layer, and further wherein the Webapplication and the Web service comprise parts of a Web server, and theweb server and the management system comprise parts of the computingdevice; in response to remote user input at the remote device that isentered using the remote graphical user interface as presented on theremote device and that is communicated by the remote device to the Webservice, providing a remote user access to the management system byproviding the remote user input to the core module via the remotinglayer and generating a control request or a data request correspondingto the received remote user input; providing a local graphical userinterface for simultaneous access of the management system by a localuser; suppressing on-screen display of messages on at least one of themedia device or the heating, ventilation, and air conditioning systemthat is accessed by the remote user; receiving a message from at leastone of the media device or the heating, ventilation, and airconditioning system acessed by the remote user; and displaying themessage via the remote graphical user interface in lieu of thesuppressed on-screen display of the messages.
 7. The non-transitorycomputer-readable medium of claim 6 wherein the non-transitorycomputer-readable medium is a memory of the computing device.
 8. Thenon-transitory computer-readable medium of claim 6 wherein the methodfurther comprises: receiving a response to a control request or a datarequest from at least one of the media device or the heating,ventilation, and air conditioning system through a gateway module of themanagement system; sending the response from the gateway module to thecore module; and communicating the response from the core module to theWeb application, via the remoting layer, to allow the response to bepresented on the graphical user interface accessible by the remotedevice.
 9. The non-transitory computer-readable medium of claim 6wherein the management system further includes a storage unit to storedata related to the control requests and data requests, and a datamanager to manage communication of data between the core module and thestorage unit.
 10. A computing system configured to control a pluralityof networked devices in an environment, the computing system comprising:one or more processors and one or more memories; and a management systemthat when executed by the one or more processors communicatively couplesto: at least one media device via a first network connection that uses afirst communication protocol, wherein the media device iscommunicatively coupled to at least one content presentation device, toa lighting system, and to at least one automated wall panel; at leastone computing system via a second network connection that uses a secondcommunication protocol, wherein the computing system is coupled to aheating ventilation and air conditioning system to allow the computingsystem to control operation of the heating, ventilation, and airconditioning system; and at least one drapes device via a third networkconnection that uses a third communication protocol, wherein thelighting system, the at least one automated wall panel, the heating,ventilation, and air conditioning system, and the at least one drapesdevice share state information including device status with each other,and wherein the lighting system, the at least one automated wall panel,the heating, ventilation, and air conditioning system, and the at leastone drapes device modify their operation based at least in part on theshared state information, the management system including: a core moduleto generate control requests and data requests related to operation ofthe at least one media device, the computing system, and the at leastone drapes device; multiple gateway modules that each correspond to adifferent type of networked device, wherein the multiple gateway modulesinclude first, second, and third gateway modules correspondingrespectively to the at least one media device, the computing system, andthe at least one drapes device, and wherein the gateway modules are incommunication with the core module to receive the control requests anddata requests from the core module, and are further in communicationwith the core module to provide responses to the control requests andthe data requests to the core module; and first, second, and thirdcommunication interfaces respectively included with the gateway modulesto provide the control requests and the data requests generated by thecore module to the at least one media device, the computing system, andthe at least one drapes device via the respective first, second, andthird network connections using the respective first, second, and thirdcommunication protocols, and to provide responses to the controlrequests and the data requests to the core module, wherein each of thecommunication interfaces is associated with the corresponding type ofnetworked device, wherein the core module and the multiple gatewaymodules are implemented as computer-executable instructions stored inthe one or more memories and executed by the one or more processors. 11.The system of claim 10 wherein the management system further includes agraphical user interface that can be used to instruct the core module togenerate the control requests and the data requests, and that canfurther be used to receive from the core module and to present at leastsome of the responses to the control requests and data requests receivedfrom the at least one media device, the computing system, and the atleast one drapes device.
 12. The system of claim 10 wherein a controlrequest related to operation of the heating ventilation and airconditioning system; as provided by the management system to thecomputing system via the second network connection using the secondcommunication protocol or a control request related to operation of theat least one drapes device as provided by the management system to theat least one drapes device via the third network connection using thethird communication protocol, is based on context information associatedwith a control request that is provided by the management system to theat least one media device via the first network connection using thefirst communication protocol and that is related to operation of the atleast one content presentation device, and wherein the contextinformation is communicated between the at least one media device andthe lighting system or the at least one automated wall panel, or betweenthe at least one media device and the computing system, the contextinformation being usable by the management system to determine whichcontrol requests related to operation of the lighting system, the atleast one automated wall panel, the heating, ventilation, and airconditioning system, and the at least one drapes device should begenerated to complement operation of the at least one contentpresentation device.
 13. The system of claim 10 wherein the lightingsystem, the at least one automated wall panel, the heating, ventilation,and air conditioning system, and the at least one drapes device modifytheir operation by optimizing their operation based at least in part onthe device status of at least one of the lighting system, the at leastone automated wall panel, the heating, ventilation, and air conditioningsystem, and the at least one drapes device.
 14. The system of claim 10wherein the lighting system, the at least one automated wall panel, theheating, ventilation, and air conditioning system, and the at least onedrapes device modify their operation by timing their operation based atleast in part on the device status of at least one of the lightingsystem, the at least one automated wall panel, the heating, ventilation,and air conditioning system, and the at least one drapes device.
 15. Thesystem of claim 10 wherein state information is maintained on the atleast one automated wall panel, the heating, ventilation, and airconditioning system, and the at least one drapes device, and wherein themanagement system does not store the shared state information.
 16. Thesystem of claim 10 wherein the management system is extensible such thatan additional gateway module that corresponds to an additional networkeddevice may be added to the management system for use with the additionalnetworked device.